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Variability  in  Complex  Scenes 


-Geometric  variability 

•  Position 

•  Orientation 

•  Articulation 

•  Fingerprint  (in  the  Bob  Hummel  sense) 

-Environmental  variability 

•  Thermodynamic  variability  in  infrared 

•  Illumination  variability  in  visual 

-Complexity  variability 

•  Mumhor  of  objects  not  known 
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Pattern  Theory:  The  Grenander  Program 


*  Representation: 

-  Incorporate  variability  in  the  parameter  space 

*  Possibly  many  nuisance  parameters 

-  Model  mapping  from  parameters  to  data 

*  Inference: 

-  Build  inference  algorithms  using  the  representation 

-  Ex.:  Markov  chain  Monte  Carlo  (jump-diffusion) 

*  General  notion:  avoid  “preprocessing”  or  “feature 
extraction”  as  much  as  possible  to  avoid  loss  of 
information 

-  Recall  Biao  Chen’s  mention  of  the  Information  Processing 
Inequality 

*  Apply  tools  of  Bayesian  inference  to  weird  things 
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Legacy  Work 


*  Sponsored  by 

-  U.S.  Army  Center  for  Imaging  Science 

(ARO  -  David  Skatrud/Bill  Sander) 

-  ONR  (Bill  Miceli) 

*  Collaborators 

-  Michael  Miller  (now  at  Johns  Hopkins) 

-  Donald  Snyder  (Washington  Univ.) 

-  Anuj  Srivastava  (now  with  Dept,  of  Stat.,  Florida  State) 

*  Airborne  targets  -  radar 

-  Matt  Cooper  (now  with  Xerox) 

*  Thermodynamic  variability  of  targets 
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Parameter  Space  for  ATR 


Parameter  space  for  a  single  target: 

X(1)  =  9?2x[0,2  jt)xA 
A  =  {M2,M60,T62...} 

Parameter  space  for  an  n-target  scene: 

X(n)  =  [y{2  x[Q,27t)xAY 


Number  of  targets  not  known  in  advance: 


00 


X  =  (J[9?2x[0,2  x)xA] 


n 


12—0 
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Inarid’s  Third  Approach 


•  Data  y,  parameters  X 

•  Likelihood  p(y  |  x) 

-  Render  infrared  scene  onto  detector  plane 

-  Model  sensor  noise  effects 

•  Prior  p(x) 

•  Bayesian  posterior 

tt{x)  =  p{x  \y)<*-p(y\  x)p(x) 


-  Analytically  forboding! 

Sample  via  jump-diffusion  processes 

-  Jump  from  subspace  to  subspace 


-  Diffuse  to  refine  estimates  within  a  subspace 
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Take  Home  Message 


Go  read 

Ulf  Grenander’s 
papers  and  books. 

They  are  very  cool. 
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Perspective  Projection 
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Sensor  Effects 


Optical  PSF 

Poisson 

Photocounting 

Noise 

Dead  and 
Saturated 
Pixels 
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FUR  Loqlikelihood 


CCD  loglikelihood  of  Snyder,  Hammoud,  and  White 

Lccoiy  I X) = -XMO + Xv(  Oin/XO 


where  //(./')  = 

j 

Cascade  with  render :  x  — »  X 


L(y  I  x)  =  LCCD(y  |  render^)) 

Sensor  fusion  natural;  just  add  loglikelihoods 
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Larmevin  Processes  Process 


•  Write  posterior  in  Gibbs  form:  7r(x)  =  CXp  {//(x)}  / Z 

•  Consider  a  fixed  number  of  N  targets  and  target  classes 

•  Simulate  Langevin  diffusion: 


dXN(r )  =  {H(Xn(T))}  +  dWN(r) 


Distribution  of  X N  (Y )  — — — — >  71 N  {xn  ) 


*  Computed  desired  statistics  from  the  samples 

*  Generalizes  to  non-Euclidean  groups  like  rotations 

*  Gradient  computation 

-  Numeric  approximations 

-  Easv  and  fast  on  modern  3-D  graphics  hardware 
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Diffusion  Example  on  AMCOM  Data 
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Helpful  Properties  of  Jump  Processes 


•  Jump  at  exponentially  distributed  times 

<2  (x)  =  {states  reachable  from  x} 

<2  (x)  =  {states  from  which  x  can  be  reached} 

•  Move  reversability:  <2  (x)  =  ?~\x) 

•  Connectedness:  can  go  from  any  point  to  any 
other  in  a  finite  number  of  jumps 


Detailed  balance  (in  discrete  form) 

;r(x)Pr(x  — >  z)  =  ;r(z)Pr(z  — »  x) 


(continuous  form  slightly  more  complicated) 
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Jumping  Strategies 


•  Gibbs 

-Sample  from  a  restricted  part  of 
the  posterior 

•  Metropolis-Hastings  style 

-Draw  a  “proposal”  from  a 
“proposal  density” 

-Accept  (or  reject)  the  proposal 
with  a  certain  probability 
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Example  Jump-Diffusion  Process 
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How  to  Model  Clutter? 


•  Problem:  Algorithm  only  knows  about  tanks 
(which  Bob  doesn’t  like  anyway),  and  will  try  to 
explain  everything  with  tanks! 

-  Cows,  swimming  pools,  school  buses 

*  Solution  (?):  Let  the  algorithm  use  flexible 
representation  in  addition  to  rigid  objects 

-  Blobs:  Simple  connected  shapes  on  the  lattice  to 
represent  structured  clutter 


-  Could  use  active  curves,  level  set  methods  (Clem 
Karl),  active  polygons  (Hamid  Krim) 

-  Clutter  might  be  interesting  in  its  own  right! 
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Random  Sampling  for  Blobs 


•  Set  of  jump  moves 

-Add  a  pixel  along  the  boundary 
-Remove  a  pixel  along  the  boundary 
-Keep  the  blob  a  blob 

•  Pick  move  based  on  posterior 
probability 
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Blob  Estimation  Examples 


M60  spreading 


M60  decaying 


Ship  decaying 
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NVESD  M60  Example 
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Saccadic  Detection 


•  Current  implementation  “births”  specific  target 
types 

•  May  be  better  to  birth  simple  shapes,  and  later 
change  them  to  more  specific  target  types  (clutter 
or  target) 

•  Example: 

-  Birth  squares 

-  Deform  into  rectangles 

-  Then  jump  to  more  detailed  targets 
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AMCOM  Data  Ex.:  Finding  Tank  1 


Initial  Detection 

Low-dimensional 

refinement 

High-dimensional 

refinement 

Equilibrium 
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AMCOM  Data  Ex.:  Finding  Tank  2 


Initial  Detection 


Low-dimensional 

refinement 


Equilibrium 
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AMCOM  Data  Ex.:  Finding  ????? 


Initial  Detection 


Low-dimensional 

refinement 
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Factory  Example 
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Unified  Algorithm 


•  Extended  jump  moves 

-  Saccadic  blob 

-  Saccadic  rigid 

-  Blob  <->  rigid 

-  Break/combine  blobs 

-  Change  rigid  target  types 


Difficulty:  Make  parameters  make 
sense  between  representation  types 
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